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STATE ASSESSMENT PROGRAM ITEM BANKS: 

MODEL LANGUAGE EOR REQUEST EOR PROPOSAL (REPs) and CONTRACTS 


This document provides recommendations for request for proposal (RFP)^ and contract language that state education agencies can use 
to specify their requirements for access to test item banks. An item bank is a repository for test items and data about those items. Item 
banks are used by state agency staff to view items and associated data; to evaluate items; to search the database of items; to inventory 
the item bank, evaluate pool sufficiency, and project future item development needs; to view statistical results; to help manage the 
flow of test development work; and to collaborate with the test development contractor on forms construction. The goal of this 
document is to help state education agencies develop appropriate language for inclusion in RFPs and contracts, language that 
assures state access to these critical functions during the contract period and ensures a smooth turnover of the item bank at the end of 
the contract to the state agency and/or a subsequent contractor. 

This document is based on the assumption that an agency does not wish to retain its existing item banking system (if any), but rather 
to acquire a new one as part of the contract. In this case any existing state items must be transferred by the vendor into the new item 
bank, as described in section 3.1 below. If, on the other hand, the agency already has an item banking system and wishes to continue 
to use it for new item development, different RFP/contract language will be needed to describe that system and the state’s 
expectations. Some of the recommendations below may be useful in helping to describe such a situation, but it is not the intent of this 
document to fully address the continued use of an existing item banking system. 

The recommendations that follow are divided into five categories; the first four cover each of the major topics that might be included 
in the item banking section of a typical RFP and contract. The fifth is a special category devoted to emerging developments in 
computer-based testing and the use of new or different item types. These emerging developments pose special challenges for item 
banking systems, specifically in storing and retrieving new kinds of information about innovative items and/or testing 
accommodations. The topics in this category span and incorporate by reference the four areas covered in the previous categories. 

Individual state testing needs will determine which of these recommendations are relevant and how they may be used in writing the 
RFP and contract. Where appropriate, examples or exemplary text (in italics in the second column) are included to illustrate the 
recommendations. Note that these are only examples, to be used selectively in conjunction with the recommendations in the first 
column. It is unlikely that any state would incorporate all of the recommendations or examples given below; to do so would likely be 
prohibitively expensive or result in disqualifying some suitable vendors. Instead, the agency should use these recommendations and 
examples as a checklist or guide to possible RFP/contract terms, adopting (or adapting) those that are relevant to its particular 
situation. Some recommendations or examples may also be used to describe additional cost options. 

* The term “request for proposal” (RFP) includes requests for information (RFIs) and other solicitations on the part of state testing agencies for testing services. 
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1. Item bank data requirements 


RFP/Contract Requirement 

Examples/Exemplary Text 

1.1 Database. Require the contractor to 
provide a single database containing all 
needed item bank components. 

The contractor must provide a database that includes all items, passages, artwork, 
resources, rubrics, and other components developed under this contract, as well as 
all legacy items, passages, etc., provided by the previous contractor. These data must 
be in a single database; that is, it is not acceptable to provide multiple databases that 
must be subsequently merged by the state or a subsequent contractor. 

1.2 Identifiers. Specify the coding scheme to 
be used to identify items and other 
components and whether multiple identifiers 
(e.g., legacy identifiers) will be required. 

Include in the item bank unique identifiers for items, passages, artwork, resources, 
scoring rubrics, and other item bank components. Include where appropriate 
“parent-child” relationships, as for example with items appearing in different forms 
on multiple tests. 

1 .3 Item metadata. Specify the metadata that 
must be included in the item bank. These 
specifications should cover items, passages, 
artwork, resources, manipulatives, scoring 
rubrics and other item bank components, and 
links between these components. Possible data 
elements are indicated in the examples on the 
right. Provide specific requirements regarding 
the formats of these elements. 

Specify whether items will be aligned to 
multiple sets of content standards or multiple 
versions of those standards. Define the status 
codes to be assigned to items as they move 
through the test development and test 
administration processes. 

If possible, provide a complete list of all data 
elements needed in the item bank and a 
glossary defining metadata and other terms 
used in describing the item bank. 

Maintain item content and classification data (e.g., grade, subject, cluster, objective) 
linking items to content standards. Include cognitive-level and depth-of-knowledge 
judgments, descriptors, word counts, reporting category, item type, and other data 
that may from time to time be required by the State. 

Maintain secure answer keys, scoring rubrics for polytomous items, and associated 
resources such as anchor papers, annotations, number of anchor papers per level, 
training sets, and qualifying sets. Include distractor rationales and references. 

Manage tracking/status data to help monitor the flow of items through the process. 
Status codes will be assigned to items based on the state definitions (attach). Include 
in the database committee and other third-party reviews (e.g., dates, ratings, 
comments), item history, versions, item clones, overlap, and other item relationships, 
public release dates, authorship, and source. Specify how items that appear in 
multiple versions are handled so the user knows which one is most current. 

Provide links between items and their associated passages. Maintain permissions 
and copyright data for items, passages, artwork, and graphics, including the holder, 
permission period(s), number of users, contact information, and whether permissions 
are different for paper-and-pencil and computer delivery. Provide code tables, such 
as content classification codes, availability codes, and status codes. Maintain 
anchor/linking designations. 
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RFP/Contract Requirement 

Examples/Exemplary Text 

1 .4 Statistical data. Specify the statistical data 
that need to be computed and stored. Provide 
clear definitions of each statistical element, 
including how they are calculated, or require 
the bidder to do so. If different Item Response 
Theory (IRT) models are used (e.g., three- 
parameter and Rasch), distinguish between the 
parameters calculated under each model. 
Specify procedures for redundant calculations 
to verify the accuracy of statistical 
calculations. If a third party will verify 
calculations, specify the procedures for 
transferring the data needed by the third party 
to carry out the verification. 

Compute and maintain statistical data from field and operational tests, including 
classical statistics, item parameters, IRT statistics, DIF statistics, etc., as defined in 
the attached document describing statistical calculations. 

Maintain bias flags indicating gender, race, or ethnic bias or orientation. 

Display item and test characteristic curves, information functions, and conditional 
standard of error curves for items, tests, and sets of items. 

The system must allow for tracking of separate statistics when an item is used in 
multiple administrations, in multiple types of tests (e.g., biology items used in other 
science tests), and for items aligned to multiple versions of content standards. 

1.5 Test construction data. Specify the data 
that need to be stored in the item bank to 
support test construction functions. 

Include in the item bank blueprint specifications for forms construction; forms data 
such as the list of items on a form, positions, keys, contribution to subscores, etc.; 
and field test design specifications, including data about the characteristics of field 
test participants. 

1.6 Items and item formats. Specify how 
items, passages, artwork, etc., should be 
formatted so that images are accessible to the 
item banking system but do not need to be re- 
authored for the publication system. Specify 
whether multiple formats or resolutions will 
be needed, for example testing 
accommodations, or whether multiple 
languages must be supported. Provide a copy 
of the state assessment unit’s publication style 
guide or other specifications that ensure 
images are formatted, used, and changed 
correctly. 

The item bank may utilize low-resolution item images and item/passage graphics to 
maximize system speed and response time, but must provide direct access from each 
item/passage record to the corresponding high-resolution files that go into the 
publication system. 

Items must be formatted in accordance with the state assessment unit’s publication 
style guide (attach a copy if one exists). 

or 

Items and item data must be provided in XML format to facilitate transitioning to and 
interfacing with other systems. 
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2. Technical requirements 


RFP/Contract Requirement 

Examples/Exemplary Text 

2.1 Database architecture. The agency should 
involve the state information technology (IT) 
department early in the process to be certain 
IT standards and RFP procedures are adhered 
to. If the state has specific requirements for 
the database architecture these should be 
spelled out in the RFP and contract. 
Architecture requirements may include table 
definitions, links between and among tables, 
and a preferred or approved database 
management system (DBMS). 

The database as delivered to the state must conform to the state ’s IT standards 
(attach a copy of the state’s IT standards if available). The DBMS must be one of the 
supported components specified in those standards. 

The database design must include links that allow all components of an item (e.g., 
text, images, metadata, statistics) to be brought together through the software so that 
the user sees the whole item, even though these components are stored in separate 
tables. 

2.2 Documentation. Specify documentation 
requirements, including technical 
specifications, data dictionaries, and 
descriptions of how data moves through the 
publications production process, how data is 
stored and retrieved for reporting, and how 
data is stored for item banking purposes. (See 
also section 4.2 for recommendations on user 
documentation.) 

The contractor must provide a complete technical manual, including table 
definitions; data formats for text, images, and metadata; links between tables; 
complete descriptions of all table elements; file naming conventions; dataflow 
descriptions and diagrams; detailed descriptions of all steps required to maintain 
and update the database; and complete technical documentation. 

2.3 Communications architecture. Specify 
whether the item banking system should be a 
web application or should run on a secure 
local area network (LAN). Specify whether 
the contractor should host the application and 
provide for remote access by state users, or 
whether the state will host the application and 
receive updated versions of the database on a 
periodic schedule. Provide a copy of the 
state’s IT communications architecture 
standards if available. 

The contractor must provide access to the item bank via a secure website. At the 
conclusion of the contract, the contractor must provide a complete system that can be 
replicated on the state ’s internal website. 

or 

The item banking system must operate on the state’s in-house network (LAN). 
Specifications for the required communications architecture are attached. 
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RFP/Contract Requirement 

Examples/Exemplary Text 

2.4 Software interfaces and interonerabilitv. 
Describe any requirements for interfaces 
among or interoperability with existing state 
office or internal software. Specify that 
materials must be provided in whatever 
format (e.g., PDF or other) the state requires. 
Describe any existing item bank or item 
banking software currently being used by the 
state, and any requirements for backward 
compatibility. Specify any standards the state 
imposes for interoperability with other 
systems. 

The item banking system must interface with Microsoft Office applications such as 
Access, Excel, and Word. Graphics must be in standard formats and mathematics 
items must be created using (specifv required software). All items and other 

images must be available in xxx format. All graphics must be re-scalable. 

The system must use only the latest versions of commercially available software and 
must use non-proprietary data formats. The contractor must specify and acquire for 
the state licenses to any run-time components or other software (e.g., DBMS) that the 
state will need to obtain. 

The item banking software must be consistent with and operate within the state ’s 
technology environment and software standards (attach a copy of the state’s IT 
standards if available). 

2.5 Svstem versioning. Specifv software 
updating requirements and whether the item 
banking system will need to be expanded over 
time to capture new types of data or to 
provide additional software functions. If the 
system is to be built in phases, describe the 
phases and require a project plan that shows 
start and end dates for each phase. Describe 
any versioning requirements for existing 
graphics files. 

The contractor must describe how the item banking software will be updated and 
remain current over time, including upgrades for operating system, hardware, and 
other technical changes. 

It must be possible to expand the item bank and software to include new data fields 
and new software functionality. 

Any enhancements and upgrades made to the item banking system provided by the 
contractor during the contract term must be applied to the state ’s item banking 
system. 

2.6 Acceptance testing. Describe, or have the 
contractor describe, the procedures for 
software and database acceptance testing. List 
any technical benchmarks that must be met 
during the acceptance test, such as system 
load, user response times, and functions the 
software must be able to perform. General 
practice is to require a response time of one 
second or less and support for as many 
simultaneous agency staff as the state is likely 
to have. 

The item banking system must accommodate xx simultaneous agency staff with 
maximum response times ofx seconds. The contractor should describe any technical 
limitations that might be imposed by the state ’s required communications 
architecture (as identified in section 2.3). 
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3. Item bank administration procedures 


RFP/Contract Requirement 

Examples/Exemplary Text 

3.1 Item bank transfer. The contractor should 
specify the procedures for loading legacy data 
at the beginning of the contract and for 
transferring the item bank to the state and/or a 
subsequent contractor at the end of the 
contract. Include in the RFP an inventory of 
existing items by format (printed, computer- 
delivered, Braille, etc.). If items must be 
recoded, for example to conform to a new 
standards hierarchy, specify the procedures 
and timeline for recoding, which items need to 
be recoded (if not all), and who will be 
responsible for carrying it out, updating the 
item bank and verifying its accuracy. 

The contractor must describe the procedures, requirements, and time needed for 
converting and loading legacy data at the beginning of the contract and verifying the 
accuracy and completeness of the transfer. 

The contractor must provide a transition plan for turning the item bank and software 
over to the state at the end of the contract. The transition plan must include 
procedures for certifying the accuracy and completeness of the item bank as 
delivered to the state and interim deliverables (e.g., sample databases) that provide 
reassurance that the item bank will meet state expectations. The transition plan will 
ensure that the state has all the knowledge and resources necessary to operate the 
item bank, and/or transition it to a new contractor, at the end of the contract. 

The end-of-contract transition plan must include the transfer of all copyrights and 
permissions from the test development contractor to the state and/or a subsequent 
contractor. 

3.2 Item bank operation and update. Specifv 
the state’s requirements and schedule for 
ongoing updating and operation of the item 
bank. Specify whether the state or the 
contractor will host and operate the item 
banking system. If appropriate, specify the 
state’s preferred mechanism for transmitting 
data, for example, on physical media (CDs), 
via secure remote hosting, through secure web 
access, or through secure file transfer protocol 
(FTP). If the state might be transitioning to 
new standards during the contract period, 
specify the conversion requirements and 
whether the costs of recoding the item bank 
should be a separate cost option. 

The contractor must describe the procedures for the physical transfer, installation, 
loading, and updating of the item bank on an ongoing basis, including procedures for 
loading statistics into the item bank after test administrations and the schedule and 
time required for delivering updates to agency staff. Updates must include an item 
inventory and item bank analyses that provide snapshots of the current status of the 
bank. 

The contractor must update statistical and tracking information in the item bank each 
time an item is used in afield test or operationally. Historical information must be 
retained for all items, including those developed prior to this contract. 

The contractor must update code tables (e.g., blueprints, content hierarchies, IRT test 
target curves, and other static tables) as needed. 
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RFP/Contract Requirement 

Examples/Exemplary Text 

3.3 Quality control (OC) and auditing 
nrocedures. Describe the requirements for 
auditing the item bank (by state staff or an 
independent consultant) and getting the 
contractor to correct errors. This should be 
done at initial item bank load or conversion, 
periodically (e.g., quarterly or yearly) during 
the contract, and at end-of-contract transfer of 
the item bank to the state. 

The accuracy of updates to the item bank will be quality controlled through manual 
inspection of sample records and summary statistics automatically generated by the 
updating system. 

The item bank must contain the latest version of the item. Images must be clean; that 
is, not scanned from old test booklets. 

QC procedures must be described for each activity on the item bank development 
plan. 

3.4 Security and backup. The contractor 
should describe their procedures for ensuring 
the security of the item bank, both during 
physical transfer and during operations. If 
appropriate, required levels of user security 
should be specified. The state’s disaster 
recovery plan should be included and the 
contractor should be required to adhere to it. 

Procedures must be included for protecting the security of sensitive data from 
general access, for ensuring that data is altered only by authorized users, and for 
restoring the item bank in case of data entry errors. Multiple levels of user security 
must be provided, for example to use the item bank for both secure and non-secure 
items and move items from one status to the other. 

The contractor must describe their provisions for off-site backup of the item bank, 
including a full backup of the item bank at least every xxx days. 

3.5 Roles and responsibilities. Define the roles 
and responsibilities of the contractor(s), any 
external consultants, and state agency staff in 
item bank administration. 

Each task on the development and operations project plan must designate the party 
responsible for its completion and the party with responsibility for reviewing the 
results. 

3.6 Ownership. Specify the state’s intention 
regarding ownership of the item banking 
system, including who will own the 
supporting applications described below 
(section 4.1). Possibilities include full state 
ownership or a license to use it for a stated 
purpose. In either case the state should retain 
full ownership of all data in the item bank 
and, if appropriate, should limit (or prohibit) 
use of items by the contractor for any other 
purpose. 

The state will own all rights to the item banking system, including source code and 
documentation. These rights include distribution and use of the software for other 
purposes. 

or 

The item bank and all of the data in the item bank will be the sole property of the 
state. The contractor will retain ownership of the software and provide to the state a 
limited perpetual license to use it, with the right to receive any future enhancements 
that the contractor makes available to other parties. 
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4. User support requirements 


RFP/Contract Requirement 

Examples/Exemplary Text 

4.1 Annlication functions. Describe the 
supporting software functions that need to be 
included in the item banking system. Some 
examples are on the right. 

Specify whether state users will need to 
update the item bank, add comments on 
individual items as they are reviewed, author 
items, or participate in assembling forms and 
generating form statistics and IRT curves. As 
agency staff becomes more involved in item 
and test construction, more of these software 
functions may be needed over time. 

Describe any standard reports that should be 
produced and the schedule on which they 
should be provided. 

Typical supporting functions: 

• item entry and editing 

• search and retrieval 

• display of items, passages, artwork, resources, and rubrics 

• annotation of items as they move through the test development process 

• tracking of permissions 

• test construction/form assembly 

• display of item and test statistics and curves 

• ad hoc report generator 

• data import and export 

• item inventory and item bank analysis and evaluation 

Users must be able to perform searches on all available data fields, including text 
phrases within text fields. Once a set of records has been found, users must be able to 
export data from all available fields to Excel and to text (e.g., csv) files. 

Users must be able to identify items that have not appeared on afield test form or 
have not been used operationally over a specified time period. 

4.2 Training and documentation. Specifv 
training needs and requirements for both 
online and printed user guides. Describe the 
procedures for authorizing and setting up 
users. Specify whether a tutorial, using a non- 
secure version of the item bank, is needed. 

User guides must at a minimum contain procedures for logging on to the item 
banking system; the basic flow and functions of the system; procedures for searching 
the database and retrieving sets of items, item worksheets, and other displays; all 
menu functions; procedures for constructing forms (if applicable); procedures for 
creating reports; data import and export functions; tools for item inventory and item 
bank analysis; and a complete data dictionary. 

4.3 Customer service. Describe the state’s 
requirements for customer service and 
technical support. Ask the contractor to 
describe its procedures for handling user 
requests for changes in system functionality. 

The contractor must provide technical support between the hours o/ (insert required 
time period including applicable time zone), Monday through Friday. 

The system must include a provision for online submission and tracking of change 
requests by authorized users. 
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5. Computer-based testing and new item types 


In addition to the requirements listed above, computer-based delivery and testing accommodations create new requirements that need 
to be spelled out in the RFP and contract. 


RFP/Contract Requirement 

Examples/Exemplary Text 

5.1 Item data and formats. Computer-based 
items and items suitable for testing 
accommodations may be composed of new 
types of data or objects. Specify the 
characteristics of the items envisioned for 
computer delivery or testing accommodations; 
for example, whether they contain audio or 
video components and how many of each. 

Items will need to be formatted for computer 
display as well as printing. Specify whether 
the same item might be simultaneously used 
in both modes and/or for accommodations. 

In addition to traditional text and graphics files, the item bank must be able to store 
audio files, video files, interactive (e.g.. Flash) objects, and test-taker tools such as 
rulers, calculators, and protractors. 

The item bank may contain multiple versions of items for alternate formats, such as 
Braille, multiple technology delivery platforms, and alternate text for screen readers. 
The alternate versions must include any descriptions or art modifications needed to 
reproduce the item in these alternate forms. The system must be able to render items 
exactly as they will appear to the test taker. 

5.2 Item metadata. New kinds of metadata 
will be needed to describe these item types 
and how they are used in the test. Specify and 
provide examples of the kinds of data the 
system will need to be able to store and 
maintain. 

Provide protocols for navigating alternative item formats and complex items such as 
simulations, for example in the form of electronically recorded storyboards. Store 
these protocols in the item bank in a form that allows agency staff to see the path(s) 
the test taker will take through the item. 
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RFP/Contract Requirement 

Examples/Exemplary Text 

5.3 Scoring and statistical data. Scoring 
rubrics may involve complex rules, especially 
for simulation tasks. Describe what is known 
about the scoring process for each unique item 
type. Specify how the item bank will manage 
statistics for multiple versions of items, e.g., 
for an item administered in both computer and 
paper-and-pencil form or used on alternative 
test forms. Describe any unique statistics that 
must be stored for new item types. 

The item bank must be able to store scoring rubrics that may be in the form of if/then 
tables or complex logic structures. The contractor should propose an interface that 
allows the user to view and validate these rubrics. 

The item bank must be able to distinguish between multiple sets of statistics for the 
same item delivered in alternative formats. 

The item bank must be able to store statistics relating to response times for 
computer-delivered items. 

5.4 Conversion to comnuter deliverv and/or 
testing accommodations. Specifv how items 
will be converted from paper-and-pencil 
format to computer-based testing or 
alternative formats. Specify the number and 
format of items that need to be converted and 
the roles of each party in the conversion. This 
is especially important when different 
contractors provide paper-and-pencil testing 
and computer delivery, or when different 
contractors provide item development and test 
delivery. 

The contractor must convert existing items formatted for paper-and-pencil 
administration to a format suitable for computer delivery according to specifications 
(attached) provided by the delivery contractor. The contractor must convert xx 
existing items to alternative formats suitable for testing accommodations. 

5.5 Software interfaces. New software 
capabilities are needed to support computer- 
based testing, including the ability to display 
different kinds of items, the ability to 
construct pools of items for computer 
delivery, and the ability to capture and store 
test results. Describe the interfaces needed 
between these new systems and the item 
banking system. 

The software must allow the user to retrieve and display/play audio files, video files, 
interactive objects, and test-taker tools. It must also be able to display the new kinds 
of metadata and scoring and statistical information described above. 

The item bank must interface with the computer adaptive testing ( CAT) pool builder 
to provide item metadata the pool builder needs and to store information about the 
resulting CAT pools (e.g., lists of items in a pool). 
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